Chapter 2 

Individual Differences in Cognition: New Methods 
for Examining the Personality-Cognition Link 


William Revelle, Joshua Wilt, and Allen Rosenthal 


Overview 

Understanding how all people are the same, some are the same, and yet none are the same is a 
fundamental challenge to personality and individual differences theorists in particular and psycholo¬ 
gists in general (Kluckhohn & Murray, 1953; Revelle, 1995). Unfortunately, there is little work that 
actually addresses the challenge of Kluckhohn and Murray. As is true for the rest of psychology, 
there is a strong trend toward fragmentation of the field of individual differences. Particularly in the 
United States, there is a tendency in personality and individual differences toward the lack of integration 
of theories of (non-cognitive) personality dimensions with individual differences in cognition. The 
chapters of this book are partly meant to rectify this shortcoming. We will do our part by reviewing 
some of the prior research on the effects of non-cognitive variables upon cognition and then intro¬ 
duce a new procedure, “Synthetic Aperture Personality Assessment” (SAPA) as a tool for exploring 
cognitive and non-cognitive aspects of personality. 

Although most of the current taxonomic research in personality emphasizes three (giant) to five 
(big) dimensions of personality, an alternative framework is to organize personality in terms of the 
four fundamental aspects of human nature that have long been subjects of psychological theory. 
Sources and descriptions of differences in Affect, Cognition, and Desire have been studied as pre¬ 
dictors and explanations of Behavior (the ACDs of B or, more simply, the ABCDs) since Plato at 
least. How people differ in what they feel, what they think, and what they want largely determines 
what they do. But to study the ABCDs requires studying them together rather than in isolation. The 
central theme of this book is the integration of individual differences in cognition with non-cogni¬ 
tive dimensions of personality. This is a beginning, but a full-fledged integration will require a 
better understanding of all aspects of the ABCDs. 


The ABCDs of Personality 

Personahty is an abstraction used to explain consistency and coherency in an individual’s pattern of 
Affects, Cognitions, Desires, and Behaviors. What one feels, thinks, wants, and does changes from 
moment to moment and from situation to situation but shows a patterning across situations and over time 
that may be used to recognize, describe, and even to understand a person. The task of the personahty 
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researcher is to identify the consistencies and differences within and between individuals (what one feels, 
thinks, wants, and does) and eventually to try to explain them in terms of a set of testable hypotheses 
(why one feels, thinks, wants, and does; Revelle, 2007). 

Early Greek philosophers explained the distinction between thoughts, feelings, and desires as 
representing the activity of the brain, the heart, and the liver. Although this biological model is now 
seen as a curiosity, the triology of mind (Hilgard, 1980) still drives current psychological theory 
(Ortony, Norman, & Revelle, 2005). Indeed, entire subfields of psychology have organized around 
what we feel (consider the journal of Emotion and the International Society for Research on 
Emotion), what we think (e.g., the journal Cognition, the International Society for Intelhgence 
Research and publications on cognitive science and cognitive neuroscience), and what we want (e.g., 
the journal Motivation, and the recently created Society for the Study of Motivation) as explanations 
of what we do. The International Society for the Study of Individual Differences includes members 
who study all four of the ABCDs and many who try to integrate pairs or triplets of these aspects. 

In an analysis of the role of emotions in man and machine that emphasized the role of emotions in 
effective functioning, Ortony et al. (2005) argued that it is necessary to consider the interaction of the 
ABCDs at three broad levels of functioning: the reactive, the routine, and the reflective. Environmental 
cues at the reactive level evoke fixed action patterns, while at the routine level they evoke action 
tendencies, which in turn elicit actions. In a negative feedback loop, these actions, in turn, reduce the 
action tendencies that evoked them (Atkinson & Birch, 1970; Revelle & Michaels, 1976; Revelle, 
1986). The reflective layer is a control layer for the two lower ones that monitors and steers the per¬ 
formance of the reactive and routine levels. This multilevel control model of the ABCDs owes much, 
of course, to the prior work of Broadbent (1971), MacLean and Krai (1973), Sloman, Chrisley, and 
Scheutz (2005) and had been proposed in a less complete form earlier (Revelle, 1993). 

The ABCDs may be used as a conceptual framework for the study of a particular personality 
dimension, (e.g., extraversion (Wilt & ReveUe, 2009) - see Table 2.1), or as a framework for integrating 


Table 2.1 Using the ABCD approach to organize personality studies: the example of extraversion. Representative 
studies have been chosen for each of the four basic components and the six “edges” of extraversion 


Component 

Study 

Finding 

A 

Lucas and Fujita 
(2000) 

A meta-analysis of 35 studies revealed an average correlation close to 
.40 between extraversion and positive affect 

B 

Paunonen (2003) 

Extraversion was positively related to alcohol consumption, parties 
attended, dating variety, and exercise 

C 

Uziel (2006) 

Extraversion was related to cognitive evaluations of hypothetical events 
as more positive but not less negative 

D 

Roberts and Robins 
(2000) 

Extraversion was related to endorsing more economic, political, and 
hedonistic goals 

Edge 

Study 

Finding 

A-B 

Fleeson, Malanos, and 
Achille (2002) 

Regardless of an individual’s trait level of extraversion, instructions to “act 
extraverted” in a group discussion increased concurrent positive affect 

A-C 

Robinson, Meier, and 
Vargas (2005) 

Among individuals scoring low on extraversion, quickness to categorize 
threatening stimuli as threatening related to experiencing negative 
affect in daily life 

A-D 

Elliot and Thrash 
(2002) 

Scales measuring extraversion, positive affect, and approach motivation 
loaded together on a higher-order factor termed “approach 
temperament” 

B-C 

Lucas and Diener 
(2001) 

Extraversion related to the cognitive interpretation of social behavior as 
highly rewarding under pleasant conditions 

B-D 

Heller, Komar, and Lee 
(2007) 

Goals categorized as approaching positive outcomes were related to 
extraverted behavioral content 

C-D 

Lieberman and 

Rosenthal (2001) 

Extraversion related to higher performance on the cognitive task 
of nonverbal decoding when individuals held conversation- 
maintenance goals 
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research across disparate fields. The four fundamental components can be analyzed separately, or as six 
pairwise “edges” (e.g., Affectx Cognition, Affect x Behavior, etc.), four “facets” of triples (e.g., 
Affectx Behavior X Cognition, etc.) or a complete integration of aU four. Examples of “edge” studies 
that include the cognitive aspect include the effect of affective biases in cognitive appraisal (Rogers 
& Revelle, 1998; Weiler, 1992), the effect of cognitive representations upon behavioral variabihty 
(Khrs & Revelle, 1986), the effect of the trait of obsessiveness upon cognitive biases (Yovel, ReveUe, & 
Mineka, 2005), and the effect of affective states upon categorization (Gasper & Clore, 2002). 


Examples of Lab-Based Studies of the ABCD “Edges” 

Affective Biases in Cognitive Processing 

Experimental personality research as well as experimental psychopathology has long been interested in 
the effect of short-term (state) and long-term (trait) differences in affect on cognitive processing (Mineka 
& Gilboa, 1998). Indeed, a classic theory of depression associates trait/state depression with a cognitive 
bias toward remembering negative events (Beck & Weishaar, 1989). Many studies of anxiety use the “dot 
probe” task (using choice reaction time measures to a dot presented following positive, negative, or 
neutral cues). Using this paradigm, it is possible to show attentional biases toward or away from threat 
that vary as a function of state and trait anxiety (MacLeod, Rutherford, Campbell, Ebsworthy, & Holker, 
2002). Trait anxiety affects the speed at which threatening faces “pop-out” of a crowd (Gilboa- 
Schechtman, Ben-Aitzi, Jeczemien, Marom, & Hermesh, 2004) and the likelihood of categorizing faces 
as threatening versus neutral after exposure to punishment (Yoon & Zinbarg, 2007). The speed at which 
people recover from an emotional induction can be assessed by the persistence of impaired reaction times 
to naming the color of emotionally valenced words (Williams, Mathews, & MacLeod, 1996) following 
an emotion induction (Gilboa & Revelle, 1994; Gilboa-Schechtman, Revelle, & Gotlib, 2000). 


Affective Versus Cognitive Processing 

Many words include both semantic and affective content. Consider which phrase in the following 
triplet does not belong (or is least similar to the other two): drown, fall down, or swim? Drown and 
fall down both have a negative valence, but drown and swim are semantic associates. On a more 
positive note, consider hug, smile, and face. Hug and smile share positive valence, but face and smile 
are semantic associates. Based upon the interpretation of Gray’s original Reinforcement Sensitivity 
Theory (Gray, 1987), Weiler (1992) showed how individual differences in sensitivity to pleasant 
(SPS) versus sensitivity to unpleasant stimuli (SUPS) were independent of each other (Table 2.2) and 
related to the tendency to classify words based upon their positive affect (sensitivity to pleasant) or 
negative affect (sensitivity to unpleasant). As would be expected, SPS was associated with 
Extraversion, Sociability, and Surgency while SUPS was related to Neuroticism (Weiler, 1992). 


State and Trait Effects on Affective Versus Cognitive Processing 

In a follow up of Weiler’s study (Weiler, 1992), Rogers & Revelle (1998) examined the effect of 
mood state and personality trait on similarity in judgments of words differing on affective and 
semantic content. Trait extraversion and trait neuroticism interacted in judging word pairs differing 
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SUPS 




-0.56 0.02 The beauty of sunsets is greatly over-rated 

-0.55 -0.06 I prefer to take my bath or shower as quickly as possible just to get it over with 

-0.51 0.09 The warmth of an open fireplace does not especially sooth or calm me 

0.51 0.11 When I pass by a bakery, I just love the smell of fresh baked breads and pastries 

0.50 -0.04 Beautiful scenery can touch something deep and strong inside me 

0.47 -0.22 I have been fascinated with the dancing of flames in a fire place 

-0.45 0.12 I don’t find anything exhilarating about a thunderstorm 

0.44 0.05 Having my back massaged feels wonderful to me 

0.18 0.52 I am always adjusting the thermostat, or wishing I could 

0.15 0.49 It is very annoying to me when a radio isn’t tuned quite right 

0.15 0.49 I find body odor extremely offensive 

0.15 0.48 I find it very disappointing when something doesn’t taste as good as I thought it would 

-0.05 -0.47 Bad odors have seldom bothered me 

0.12 0.46 Even the smallest piece of gravel in my shoe just drives me crazy until I can get it out 

-0.09 0.44 I have terrible feelings when I am not sure that I will succeed 

0.31 0.42 It is important to me to get the water temperature just right when I take a bath or shower 


in affective content. High levels of neuroticism were associated with judging affectively negative 
word pairs as more similar than affectively positive pairs, but only for high levels of extraversion. 
More extraverted participants judged positive word pairs as more similar than negative pairs, but 
only if they were low on neuroticism. State Negative Affect was associated with faster response 
times in categorizing negative words than positive words, even when trait extraversion and neuroti¬ 
cism were controlled statistically. 

But stimuli need not be valenced or threatening to be affected by affective state. When judging 
the similarity of objects, one can use global or local characteristics to make the judgments. Induced 
positive affect increases the use of global cues while induced negative affect increases the use of 
local cues (Gasper & Clore, 2002). 


Individual Differences in Cognitive Representation and Behavioral Variability 

A fundamental finding from cognitive psychology is that behavior is a function not of the objective 
environment, but of the environment as perceived and cognitively structured. How individuals 
organize their views about their world (their life space) determines their behaviors in the actual 
world (Lewin, Adams, & Zener, 1935). As an elegant example of this concept, Wish, Deutsch and 
Biener (1970) found that individuals categorized nations along several independent dimensions: 
e.g., developed-less developed, communist-noncommunist, northern-southern. Using a program 
for scaling individual differences in multidimensional scaling, INDSCAL (Carroll & Arable, 1980; 
Carroll & Chang, 1970), Wish et al. (1970) found that individual differences in how much these 
dimensions were weighted predicted attitudes toward the Viet Nam War. Those who weighted the 
developed-undeveloped dimension more were much more in favor of withdrawing from Viet Nam, 
while those who weighted the communist-non communist dimension more were in favor of contin¬ 
ued hostilities. 

At a more individual level, an INDSCAL analysis of stressful situations showed that college 
students reported greater consistency of behaviors between situations that they thought of as similar 
(based upon their personally weighted multidimensional space) rather than situations judged as 
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similar by the entire group (Klirs & Revelle, 1986). Individual differences in breadth of cognitive 
processing are related to obsessive compulsive behaviors (Yovel et al., 2005). Highly obsessive 
individuals (as judged by self report) are much more hindered by details when asked to do a speeded 
classification task of large letters made up of conflicting smaller letters using the “forest-trees” task 
of Navon(1977). 

These studies of the “edges” of the ABCDs are merely examples of the ways in which individual 
differences in affect, cognition, and desires affect behavior. They are all examples of what can be 
done in lab-based studies and are thus limited in sample size as well as generalizability. In the rest 
of this chapter, we consider a new technology for studying individual differences in the ABCDs that 
does not have this limitation. 


Synthetic Aperture Personality Assessment: Using the Web for Data 
Collection 

Studies of individual differences in cognitive and non-cognitive aspects of personality are frequently 
limited by the sample sizes available in the typical university research setting. Small but stable rela¬ 
tionships are difficult to detect when one is limited to 50-100 subjects, and detecting complex relation¬ 
ships between multiple measures is difficult when participants are limited to short 1 or 2-h studies. 
Alternative procedures might involve large research groups collecting data across many research sites 
(e.g., the Programme for International Student Assessment - PISA), but these can be costly and time 
consuming to conduct. A relatively new procedure is to use web-based data collection techniques to 
increase the sample size both numerically as well as in breadth with little loss of validity (Fraley, 2004; 
Gosling, Vazire, Srivastava, & John, 2004; Skitka & Sargis, 2006). Although some very large samples 
are available this way (e.g., the >300,000 reported by Gosling et al.), the studies are typically limited 
to short questionnaires or basic cognitive tasks (Greenwald, Nosek, & Banaji, 2003). 

A variation of standard web-based assessment methods is to borrow by analogy a technique used 
in radio and optical astronomy: Synthetic Aperture Measurement. The resolution of a telescope is 
limited by its diameter which may be functionally increased a great deal by combining input from 
multiple, linked sites into one coherent image. Effectively, a very large telescope is created by syn¬ 
thesizing the input of many smaller ones. A classic example in radio astronomy is the Very Large 
Array in Socorro, New Mexico where 27 relatively small (*25 m) radio telescopes are spread out 
in a Y-shaped configuration to simulate the resolution of a 36 km telescope. The configuration is 
adjustable so that the telescope can either emphasize resolution (by maximizing the distance cov¬ 
ered) or sensitivity (by concentrating the telescopes close to each other). Similar techniques are used 
in optical inferometry at the Keck Observatory in Hawaii with “outriggers” to supplement the main 
telescope. 

These techniques are available for psychologists taking advantage of the internet and web browsers. 
Rather than combining signals from the same source using different telescopes as is done in astronomy, 
the structure of personality can be studied by combining the responses of many people across more 
items than any one person is willing to take.' This, not actually a new procedure for the Educational 
Testing Service, has long used the very large samples available when students take the SAT or GRE 


‘The analogy is not perfect, for in astronomy the synthetic aperture technique provides a clearer image of one object, 
but when done by synthesizing covariance matrices, the higher resolution is applied to the structure of the measures, 
rather than to any individual. 
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to develop new items by randomly giving small subsets of items to much smaller (but still quite 
large, e.g., 1,000) subsamples of students. Now, by using open source and public domain software, 

this procedure is available to all of us. 

The basic procedure is very simple. From a large set of personality and ability items (F>400), a 
smaller subset of items (na; 60-75) are presented to any one subject. With random sampling of the 
items, all possible pairs of P*{P- 1)/2 are eventually presented. As the number of subjects grows 
(currently A> 84,000), each item has been given to N*(nlPy subjects, and each pair of items has 
been given to subjects. 


The SAPA Methodology 

Item Pool 

During the past century, the measurement of personality and ability has tended to be fragmented by 
separate groups of individuals using proprietary sets of measures. Indeed, the proprietary nature is 
partly seen in the choice of names for these inventories and tests: the Miimesota Multiphasic 
Personality Inventory, the California Personality Inventory, the Eysenck Personality Inventory/ 
Questionnaire/Profiler, the Freiburg Personality Inventory, Guilford-Zimmerman Temperament 
Survey, the Hogan Personality Inventory, the Jackson Personality Research Form, the Meyers-Briggs 
Type Inventory, the Stanford-Binet, the Wechsler Adult Intelligence Scale, the Wechsler Intelligence 
Scale for Children, etc. Each of these tests was carefully developed by researcher groups, and each is 
protected by copyright. Although some groups allow non-profit use of the measures for minimal cost, 
this is the exception. Many of these inventories have similar sounding scales, but given the expense, 
there are a limited number of studies directly comparing the inventories (Grucza & Goldberg, 2007). 

A welcome alternative to the proprietary nature of personality measurement is the International 
Personality Item Pool (IPIP) developed by Lewis Goldberg (1999). Including more than 2,400 items 
in the form of sentence stems, the IPIP collaboratory has at least 269 scales targeted at everything 
from achievement striving to vitality/zest and includes particular scales designed to provide public 
domain scales meant to be parallel to those found in at least 17 commonly used personality inven¬ 
tories. All of the IPIP items and the common personality inventories have been given to the same 
community sample in Eugene/Springfield, Oregon, and item statistics are freely available from 
Goldberg and his associates. Some have questioned the open and free use of the IPIP items with 
respect to the possibility of the public learning to fake personality tests used in employment settings. 
Others have worried about whether the freedom to select items will lead to fragmentation of con¬ 
structs rather than the hoped-for integration (Goldberg et ah, 2006). 

Even more proprietary than non-cognitive personality scales and items are measures of intellectual 
ability. Items and scales are either under copyright or completely idiosyncratic to particular labs and 
usually not openly published. Finding open source measures of ability is very difficult. Thus, to 
create a set of open source, public domain ability items, it is necessary to develop and validate our 
own. The hope is that this endeavor will inspire others to add items to the basic pool. As of now, 56 
ability items have been constructed by writing items to measure vocabulary, verbal and mathematical 
reasoning, and abstract reasoning using geometric analogies. The analogies were constructed following 
the principles discussed by Mulholland, Pellegrino, and Glaser (1980) and involved the varying 
levels of memory load by varying the number of transformations between elements of the analogy 
(Leon & Revelle, 1985). The geometric analogy items appear somewhat similar to items from a 
Raven’s progressive matrix (Raven, 1989). Plans are to develop more items and add them to the 
total pool. Here, we report an initial validation of these items against themselves as well as against 
self-reported measures of scholastic ability. 
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Subjects 

The Personality Project (http://personality-project.org) is part of an effort to increase the scientific 
literacy and understanding of personality theory of the general population. The Personality Project, 
and its affiliated site, Great Ideas in Personality which was developed lovingly by G. Scott Acton 
until he died, provide information about personality theory and research for the interested web surfer. 
The roughly 1-3,000 daily visitors to these two sites see a small notice about a web-based personality 
test that offers personal feedback. On the Personality Project this is just a single line on the first page, 
on the Great Ideas site, this is one of many tests listed in a section on personality tests. 

From these 1-3,000 visitors, as well as those who come from the results of online search engines, 
about 100 per day visit the site http://test.personality-project.org. That is, about 36,000 people per 
year flow through the SAPA procedure. As discussed in reviews of web-based research (Fraley, 
2004; Gosling et al., 2004; Skitka & Sargis, 2006), the participants are demographically diverse but 
not a representative sample of anybody other than those who want to take web-based surveys. They 
are 70% female, with a median age of 23 (Table 2.3). However, they are probably more diverse than 
participants available through any means other than an international random survey. Roughly 1/3 of 
the participants have completed less than 14 years of schooling, 1/3 are attending college, and 1/3 
have finished college. 1/6 have completed a graduate or professional degree (Table 2.3). Although 
roughly 4/5 of the participants are from North America, 16 countries (representing almost 92% of 
the total) have more than 250 participants each (Table 2.4). 


Software and Hardware 

The testing site, as well as the Personality Project and Great Ideas sites, is maintained on two Apple 
Macintosh G4 desktop computers^ running an open source web server application, Apache. The code 
for the test is written in two open source languages, PHP and HTML and makes use of a powerful 


Table 2.3 Age and education statistics of the first 80,471 subjects 



All participants 

Males 

Females 

Age 

Minimum 

10 

10 

10 

First quartile 

19 

19 

19 

Median 

23 

23 

24 

Third quartile 

34 

34 

34 

Maximum 

99 

99 

99 

Mean 

27.56 

27.37 

27.65 

10% trimmed mean 

25.91 

25.61 

26.06 

Education 

Less than 12 years 

13 

15 

12 

High school graduate 

8 

9 

7 

Some college, did not graduate 

10 

10 

10 

Currently attending college 

34 

32 

36 

College graduate 

17 

16 

18 

Graduate or professional degree 

17 

18 

17 

N 

80,471 

25,476 

55,045 


^These machines are not particularly powerful and as of this writing are three generations older than what is currently 
available. That is, the SAPA procedure is not computationally intensive. 
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Table 2.4 Although 80% of the participants are from North America, substantial 
numbers of participants report coming from other countries. Of the 207 countries 
represented, 92% of the participants come from those 16 countries with 250 or 
more respondents. There are 38 countries with more than 100 respondents 


Country 

Count 

Cumulative 

Percent 

Cumulative % 

USA 

59,792 

59,792 

74.30 

74.30 

Canada 

4,180 

63,972 

5.19 

79.50 

UK 

2,731 

66,703 

3.39 

82.89 

Australia 

2,305 

69,008 

2.86 

85.76 

India 

1,169 

70,177 

1.45 

87.21 

Philippines 

505 

70,682 

0.63 

87.84 

Malaysia 

412 

71,094 

0.51 

88.35 

China 

411 

71,505 

0.51 

88.86 

Mexico 

394 

71,899 

0.49 

89.35 

Sweden 

389 

72,288 

0.48 

89.83 

Germany 

370 

72,658 

0.46 

90.29 

Singapore 

369 

73,027 

0.46 

90.75 

Poland 

289 

73,316 

0.36 

91.11 


(and also open source and free) database program, mySQL. All analyses reported are done using the 
open source and publicly available statistics and data analysis language R (R Development Core 
Team, 2007). All of the code for this project is available from the senior author. 


Procedure 

When a participant arrives at the first page of the SAPA personality inventory, they are given a brief 
welcome screen, and then asked to agree to a consent form. The next page asks some basic demographic 
information (age, sex, education, and country of residence). If they are from the United States, they are 
then asked about their ethnic identity and if they have taken either or both of the SAT and the ACT. If 
so, they are asked to report their SAT Verbal and Quantitative scores, and their ACT total score.^ 

Following the demographic pages, participants are given 60 personality items with a six point response 
format ranging from “very inaccurate” to “very accurate.” For Study 3 (see below), this was then followed 
by 12 music preference items. For studies 3-6 and ongoing, they are then given 14 ability items. 

When participants finish all of the above items, they are then given feedback on their personality 
scores. This feedback is adapted from that given by John Johnson on an alternative (and longer) 
web-based Big 5 inventory (Johnson, 2005). Scores are reported as means (on the 1-6 scale for the 
items) as well as percentile equivalents (estimated from earlier data). The results are presented 
numerically, graphically, and in paragraph form. The paragraphs distinguish between high, medium, 
and low scores. Participants are given a personalized URL with their scores that they are encouraged 
to put into their own personal web page or blog (which then entices more participants to the site). 

The personality items used in the SAPA project so far mainly represent a subset of the 2,400 
International Personality Item Pool (IPIP) items made available by Goldberg (1999), with a particular 
emphasis on those used to assess “Big 5” dimensions as well as detailed studies focusing on particular 
content domains. Although the ideal case would sample items at random from the entire set, it was 
necessary to be somewhat systematic in order to recruit participants. Each set of items includes 50 


’For the first year, the questions about SAT and ACT were not given. For the first part of the second year, just total 
SAT was requested, but since January, 2006, all participants were asked for their SAT V and SAT Q scores. 
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items (sampled from 100) that were developed as markers of five personality dimensions, “the Big 5” 
(Goldberg, 1990; Goldberg et al., 2006), and an additional 10-20 exploratory items sampled from 300 
to 400 items of current interest. Scores on the “Big 5” items are reported using paragraph descriptors 
developed by John Johnson at Pennsylvania State University for another web-based survey and using 
norms developed locally. 


Analytical Techniques for SAPA Data 

As is obvious from the description of the data collection, there are no participants with complete data. 
Thus, descriptive statistics, correlation matrices, factor analyses, cluster analyses, and regressions are 
based upon pairwise rather than casewise deletion of subjects with missing data. Given the sampling 
design, some pairs of items have far more observations than do other pairs. AU stmctural analyses (factor 
analysis and regression analysis) were done on the pairwise deleted correlation matrix. Intercorrelations 
between scales were calculated by synthetically forming the within and between scale correlation matrices 
from the composites of the raw item correlation matrices. Functions were developed for the R computing 
environment (R Development Core Team, 2007) to do these operations on the synthetically combined data 
matrices. Many of these functions are available to the R user in the psych package (Revelle, 2008), avail¬ 
able from the Comprehensive R Archive Network (CRAN) http://cran.r-project.org website. This website 
serves as a repository for more than 1,300 packages that have been contributed to R. 

The multidimensional structure of the personality and ability items was investigated using both 
factor analysis and cluster analysis technique of the composite matrices. Principal Axis factor analy¬ 
sis was done using the factor.pa function and cluster analysis was done using the ICLUST function. 
The ICLUST algorithm (Revelle, 1979) has been adapted to R and is included as part of the psych 
package. Originally, it was developed with the specific goal of dimensional reduction of “messy” 
matrices such as those found with personality or ability items. The algorithm is similar to most 
hierarchical clustering algorithms in that it: 

1. Forms a matrix of proximities (correlations). 

2. Finds the most similar pair. 

3. Combines this pair if the pair would be better (in terms of alpha and beta) than each part. 

4. Repeats steps 2 and 3 until no pairs meet the criterion. 

ICLUST differs from many clustering algorithm in that it stops clustering when the internal consistency 
estimates (either the a or /? coefficients) fail to increase, a is an estimate of internal consistency based 
upon the average inter-item correlation as well as the number of items (Cronbach, 1951), /? is an 
estimate of the worst split half reliabihty of a test and is an estimate of the general factor saturation of 
the test (Revelle, 1979). Zinbarg, Revelle, Yovel, and Li (2005) compare these two estimates with yet 
another estimate of the general factor saturation, cOj^ (McDonald, 1999), and conclude that cOj^is superior 
in most cases, although /? is useful as a criterion in clustering applications. Revelle & Zinbarg (2009) 
consider these and eight other estimates of reliabihty as well. 


Data Cleaning 

A typical problem in web-based surveys is to distinguish legitimate unique responders from people 
who are trying the test multiple times. Because of concerns about confidentiality, some identifying 
information (e.g., MAC numbers of the computer or TCP/IP numbers for the network connection) 
are not collected. Participants are asked if they have taken the test before, and if so, are excluded 
from the subsequent analysis. To detect multiple responses from the same user over a brief period 
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of time, a random identification number is generated and stored for the duration of the connection. 
Only the first record of information with this unique number is processed. In addition, screening is 
done for similar patterns of responses across all the items (Johnson, 2005). Given the random nature 
of the items presented, it is unlikely that two people will get the exact same patterns of items and, 
if they do, even less likely that they will agree in almost all their responses. However, participants 
who respond to the questions, get their feedback, and then see what will happen if they change a 
few items that are detected this way and are excluded. Finally, participants with reported ages less 
than 10 or more than 100 are considered to have been deceptive and are rejected. 


Personality and Ability as Assessed by the SAPA Methodology 

The use of the SAPA procedure is an ongoing project of the Personahty, Motivation, and Cognition 
Laboratory at Northwestern University. Here, we hriefly outline results from seven different studies that 
have been conducted so far. The first four studies are relatively smaller demonstrations of the power of 
the SAPA technique to quickly focus on a particular target construct and will be summarized briefly. 
We spend considerably more time discussing the power of SAPA as shown in the last three studies. 


SAPA Can Provide High Resolution of Particular Traits 

Study 1: Proof of Concept: Right Wing Authoritarianism'* 

The study of the Authoritarian personahty (Adorno, 1950) was particularly active immediately following 
World War 11 but fell out of favor in the 1960s and 1970s. More recently, the concept has become a topic 
of study in terms of Right Wing Authoritarianism-RWA, (Altemeyer, 1988) which is seen as a tendency 
to be hierarchical, conventional, and intolerant (Butler, 2000). Prior work has shown systematic (nega¬ 
tive) correlations with opermess and positive correlations with social dominance. As a demonstration and 
proof of concept of the SAPA technique, we examined whether the findings from these prior studies 
could be rephcated in a web-based study. The answer was a clear “yes.” For the first »2,500 participants 
sampled, the items of the Big 5 allowed for a recovery of five dimensions, and the pattern of correlations 
with RWA matched that of prior studies. RWA correlated .23 with Conscientiousness and -.33 with 
Openness (Revelle & Laun, 2004). 


Study 2: Personality, Music Preference and Cognition® 

The second study used the SAPA method to examine the relationship of personality dimensions 
with music preferences, and introduced the study of cognitive ability into the SAPA procedure. Prior 
work (Rentfrow & Gosling, 2003) had shown systematic differences in preferences for various 
musical genres. In terms of interpersonal behavior, musical preferences are one of first things people 
discover about each other in social interactions (Rentfrow & Gosling, 2006). Measuring musical 
preferences seemed to be a very logical extension of the SAPA procedure. 


‘‘Conducted as part of an honors thesis by Gregory Laun. 

Participants numbering 2,557 were collected as part of an honors thesis by Melissa Liebert, subsequent data have 
been collected as a continuing part of the SAPA project. 
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Sixty musical preference items were presented, representing the 14 genres included in the Short Test 
of Music Preferences (STOMP) developed by Rentfrow & Gosling (2003) and incorporating additional 
items adapted from Title & Zuckerman (1986). 52,065 subjects responded to 12 item samples of the 60 
music items. Because there were large gender differences in endorsement frequencies for some genres 
(e.g., women particularly hked Broadway musicals and TV soundtracks; men particularly hked heavy 
metal) factor and cluster analysis were done for males and females separately. As is true for all correla¬ 
tion or covariance matrices, determining the optimal dimensionality is more art than science. Cluster 
analysis solutions using ICLUST showed a single cluster for men and four clusters for women. Using 
the Very Simple Structure criterion (which compares goodness of fit of solutions of progressively more 
complex structure (Revelle & Rocklin, 1979)), it was clear that music preferences were not simple 
structured and that the best solution was one of complexity two for six factors. (Complexity reflects the 
number of non-zero loadings per item. Thus, a complexity one solution attempts to recreate the cor¬ 
relation matrix from a simple structure factor matrix where all except the largest loadings are set to 0. 
A complexity two solution sets all but the largest two to zero.) Considering complexity one solutions, 
only three broad factors showed substantial factor congruence across gender. These factors then broke 
down into more complicated solutions within gender. The three broad factors could be interpreted as 
representing (1) classical, folk, and jazz, (2) rock, and (3) popular/easy listening. 

These three music factors were then correlated with the personality and demographic data. The 
classical, folk, and jazz items were most related to openness (.34), age (.30), agreeableness (.25), 
and education (.25). Preference for the rock items were negatively correlated with age (-.25) and 
positively with openness (.15). Preferences for popular and easy listening music was correlated with 
agreeahleness (.33), gender (females preferred it more, r=.26), conscientiousness, (.19) and extra¬ 
version (.17). At the item level, of the 323 IPIP, ability, and music items, the single item most cor¬ 
related with gender was a preference for Broadway musicals (r=.28). 


Study 3: Measurement of Trust and Trustworthiness'’ 

Trust and trustworthiness are essential elements of social interaction. It is difficult to conceive of 
daily life without exhibiting trust in others. The detection of cheating is important for humans as 
well as fish (Bshary & Grutter, 2002, 2006). To what extent are these two essential concepts repre¬ 
sented in standard measures of personality? In a two-part study, Evans & Revelle (2007) examined 
the factorial structure and correlations with Big 5 measures using SAPA technology, and then vali¬ 
dated their scales using an experimental procedure known as the Investment Game (Berg, Dickhaut, 
& McCabe, 1995; Bohnet & Croson, 2004). With A=8,183, Trust and Trustworthiness correlated 
highly with each other (.50) but did show differential patterns of correlations with Big 5 scales: 
Trust correlated positively with Agreeableness (.65), and Extraversion (.58), and negatively with 
Neuroticism (-.61). Trustworthiness correlated positively with Conscientiousness (.60) and 
Agreeableness (.62). Multiple regression showed that trust was best predicted by agreeableness and 
negative neuroticism while trustworthiness was predicted by agreeableness and conscientiousness. 


Study 4: Measurement of Machiavellianism’ 

In a follow up study, examining the basis of trusting behavior in the Investment Game setting, the 
trust items from Study 3 were supplemented with items taken from Machiavellianism (Mach) scales, 
which are intended to measure a person’s wilhngness to manipulate others (PauUius & Williams, 2002). 


'^Conducted as part of an honors thesis by Anthony Evans. 
’Conducted as part of an honors thesis by Samantha Holland. 
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With N> 16,000 participants, Mach items were shown to have a two dimensional structure. Factor 
1 was related to the traditional definition of Mach, the tendency to manipulate and deceive others 
for personal gain. Factor 2 was related to the belief that the true nature of other people is basically 
self-serving and unethical. These factors were labeled “Inner Mach” and “Outer Mach,” respec¬ 
tively, in order to capture the contrast between the first factor’s emphasis on the self and the second 
factor’s emphasis on others. 


SAPA Can Resolve Broader Traits 

The following three studies will be presented in more detail in order to illustrate how the SAPA 
technique can be applied in a variety of ways. The first study shows how the synthetic “telescope” 
can be focused to address questions pertaining purely to personality theory, while the second 
generalizes the technique to answer questions about cognitive ability. The last study reviews the 
findings concerning the overall structure of Big 5 scales as they relate to each other, to ability, 
and to various demographic characteristics. 


Study 5: Measurement of Extraversion Facets 

The higher-order trait dimension of extraversion has been identified as one of the fundamental 
dimensions of personality through biological and taxometric approaches (Costa & McCrae, 1992; 
Digman, 1990; Eysenck & Himmelweit, 1947; Eysenck, 1970, 1973; Goldberg, 1990; Norman, 
1963). However, disagreements about how to best characterize the core of extraversion and its facets 
remain prominent in the personality psychology literature (Ashton, Lee, & Paunonen, 2002; Costa 
& McCrae, 1998; DeYoung, Quilty, & Peterson, 2007; Hofstee, Raad, & Goldberg, 1992; Lucas, 
Diener, Grob, Suh, & Shao, 2000; Watson & Clark, 1997; Wilt & Revelle, in press). One possible 
reason that consensus definitions for the core and facets of extraversion have not been achieved is 
that most items used to measure extraversion comprise a mixture of affective and behavioral com¬ 
ponents (Pytlik Zillig, Hemenover, & Dienstbier, 2002) that fail to delineate extraversion into con¬ 
ceptually distinct facets reflecting purely positive affect, behavioral activity, and the desire for 
social attention. Study 5 is an illustration of how to use SAPA methodology to construct and evalu¬ 
ate personality facet scales measuring positive affect, behavioral activity, and desire for social atten¬ 
tion. We selected items that seemed to be pure measures of each distinct facet by searching through 
the items used to measure extraversion in the IPIP (Goldberg, 1999), which contains items targeted 
to measure the most commonly used extraversion scales such as the NEO instruments (Costa & 
McCrae, 1992), the Abridged Big-Five Circumplex (AB5C) (Hofstee et al., 1992), and the newly 
developed Big-Five Aspect Scale (DeYoung et al., 2007). Items used to measure each facet are 
shown in Table 2.5. As of this writing, over 16,000 subjects have taken these facet scales. From the 
synthetic correlation matrix, we evaluated the properties of the new facet scales. The facets were 
highly correlated with each other and each facet had high internal consistency (Table 2.5). The 
general factor saturation of the items was estimated by an cOj^ of 0.55, indicating that a common 
latent variable (hypothesized as extraversion in this case) accounted for a 55% of the variance of the 
items (Revelle & Zinbarg, 2009; Zinbarg et al., 2005). cOj^ is found by extracting a general, second 
order factor from the obliquely rotated first order factors, and then through a Schmid-Leiman trans¬ 
formation, finding the amount of item variance accounted for by that general factor. The lower order 
factors that emerged closely resemble the hypothesized structure of the facet scales, as items in each 
scale generally had their highest loading on the appropriate factor (see Table 2.5). 
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Table 2.5 Items measuring positive affect, behavioral activity, and desire for social attention 


facets and their factor loadings on three oblique factors 


Facets and Items 

Factor 1 

Factor 2 

Factor 3 

Positive affect 

Laugh a lot 

.80 

-.11 

.01 

Have a lot of fun 

.78 

-.02 

.06 

Express childlike joy 

.66 

-.16 

.17 

Radiate joy 

.62 

.10 

.21 

Laugh my way through life 

.60 

-.06 

.16 

Feel that I have a lot of inner strength 

.54 

.26 

-.21 

Feel healthy and vibrant most of the time 

.47 

.25 

.09 

Have great stamina 

.16 

.47 

-.10 

Behavioral activity 

Try to lead others 

-.10 

.86 

.13 

See myself as a good leader 

-.05 

.84 

-.05 

Have leadership abilities 

-.07 

.78 

.08 

Automatically take charge 

.08 

.69 

-.04 

Can easily push myself forward 

.27 

.55 

-.24 

Have a strong personality 

.03 

.48 

.26 

Maintain high energy throughout the day 

.35 

.41 

-.10 

Am usually active and full of energy 

.55 

.40 

-.01 

Wait for my turn (R) 

.34 

.01 

-.55 

Desire for social attention 

Like to attract attention 

.05 

.08 

.78 

Demand to be the center of interest 

-.02 

-.08 

.71 

Enjoy being part of a loud crowd 

.21 

-.06 

.71 

Can’t do without the company of others 

.08 

-.07 

.55 

Love large parties 

.17 

.23 

.54 

Usually like to spend my free time with people 

.14 

.12 

.49 

Boast about my virtues 

-.25 

.11 

.43 

Rarely enjoy being with people (R) 

.22 

.28 

.39 

Don’t like crowded events (R) 

.27 

.16 

.36 

Am afraid to draw attention to myself (R) 

.11 

.29 

.32 

Would not enjoy a job that involves social 
interaction (R) 

.24 

.26 

.26 

Like to amuse others 

.34 

.07 

.15 

Act comfortably with others 

.31 

.32 

.08 


To examine the convergent and discriminant validity of the facets, we found the correlations 
between each facet and each Big 5 domain scale. Table 2.6 shows that the social attention facet had 
very high convergent and discriminant validity, as indicated by high correlations with the extraver¬ 
sion scale but not the other Big 5 scales. The positive affect and behavioral activity facets also had 
high convergent and discriminant validity, as both of these facets correlated more strongly with the 
extraversion domain scale than any other Big 5 domain. These results could be interpreted as mean¬ 
ing that the social attention facet was a better marker of extraversion than the positive affect and 
behavioral activity facets. However, the pattern of correlations between the extraversion facets and 
the Big 5 domains may be unique to the item pool used to measure extraversion facets in this study. 
If items were drawn from extraversion scales emphasizing affective content, such as the 
Multidimensional Personality Questionnaire (Tellegen, 1982) or behavioral content, such as the 
Eysenck Personality Inventory (Eysenck & Eysenck, 1968), it is possible that the positive affect or 
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Table 2.6 Extraversion facet scales and Big 5 domain scales. Number of items in each scale, scale reliabilities, and 
correlations between extraversion facet scales and Big 5 domain scales are shown 


Scale 

N 

Cronbach’s a 

Positive affect 

Behavioral 

activity 

Desbe for social 
attention 

Positive affect 

8 

.84 

- 

- 

- 

Behavioral activity 

9 

.83 

.56 

- 

- 

Desire for social attention 

13 

.85 

.56 

.55 

- 

Extraversion 

20 

.93 

.62 

.72 

.83 

Agreeableness 

20 

.90 

.48 

.30 

.35 

Conscientiousness 

20 

.92 

.22 

.32 

-.03 

Emotional stability 

20 

.93 

.37 

.28 

.08 

Openness 

20 

.83 

.30 

.42 

.15 


behavioral activity facets would have emerged as the better markers of extraversion than the social 
attention facet. Summing across analyses, the new facet scales seem to be a generally good but not 
optimal way to measure extraversion and its distinct components. Future efforts to improve the 
scales should focus on raising the discriminant validity of the positive affect and behavioral activity 
facets and determining whether additional facets (such as a cognitive facet) should be added to the 
scales. The SARA technique provides another advantage pertinent to improving scales, as it is 
“easy” to add and subtract items from the active item pool using RHP, allowing for increased flex¬ 
ibility and adaptation in scale construction. 


Study 6: Public Domain Assessment of Ability 

Although originally developed as an open source procedure for studying non-cognitive aspects of 
personality, the SARA procedure has been applied to studying individual differences in cognitive 
ability. Starting in 2005, each participant was given 14 items thought to measure cognitive ability. 
These items were sampled in two sets of seven from a total pool of 56 items developed by Liebert 
(2006). The items were written to test alphanumeric pattern recognition, general knowledge, vocab¬ 
ulary, logical reasoning, and spatial reasoning. Example items from this set are in Table 2.5 and 
Fig. 2.1. In addition to these 14 items, any participant from the United States was asked if he/she 
had taken either the SAT or ACT exam, and if so, to report her/his scores. The hope was to be able 
to validate the new items against each other as well as against the (self-reported) standardized tests. 
Prior work has shown that self-reported SAT scores are highly correlated with actual scores, 
although self-reports are somewhat inflated (Kuncel, Crede, & Thomas, 2005; Mayer et al., 2007). 
Basic descriptive statistics for these standardized tests (Table 2.7) are remarkably similar to actual 
scores for college undergraduates (Mayer et al.). 

For the SARA procedure to be useful in assessing ability online, the items need to show basic 
psychometric properties. They need to span the difficulty range, need to correlate with each other 
and to correlate with known markers of ability. The results so far suggest that we were successful 
in all of these objectives. Item difficulties (percent correct) ranged from .15 to .96 with a mean of 
.58, a median of .62, and the first and third quartiles of .40 and .77. The average intercorrelation of 
all 56 items was .08. The average correlations with SAT, SATV, SATQ, and ACT were .12, .10, .12, 
and .12. All of these values are for unselected items. The results are much more promising when 
basic item analysis is done. Because exploratory work suggested that 1 parameter (Rasch) or 2 
parameter Item Response Theory estimates (Embretson & Reise, 2000) were not particularly better 
than simple sum scores, we report the sum scores analysis. 
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Fig. 2.1 An example of a geometric reasoning problem. Each problem differs in the number of elements and number 
of transformations per element. The participant is to choose the response that replaces the ? 


Table 2.7 Descriptive statistics for the self-reported SAT, SATV, 
SATQ, and ACT 



N 

Mean 

Sd 

Median 

AW participants 

SAT 

3,378 

1,194 

215 

1,200 

SATV 

10,987 

613 

110 

620 

SATQ 

10,852 

606 

112 

610 

ACT 

16,020 

25.6 

5.0 

26 

Males 

SAT 

1,357 

1,214 

223 

1,210 

SATV 

4,285 

613 

111 

620 

SATQ 

4,248 

622 

114 

640 

ACT 

4,845 

25.9 

5.4 

26 

Females 

SAT 

2,021 

1,180 

208 

1,190 

SATV 

6,702 

614 

110 

620 

SATQ 

6,604 

595 

109 

600 

ACT 

11,175 

25.5 

4.9 

26 


There are a number of ways to analyze these scales. Each participant took two 7-item scales. All 
possible pairs of scales (i.e., 14 items each) were then given across participants. Although each 
person took 14 ability items, the intercorrelations of these scales cannot be found directly from the 
data or can be calculated synthetically. The analysis can be done at the scale level (7 or 14 items per 
scale) or the overall covariance structure level (56 items). For both analyses, it is possible to validate 
the scales or overall factor structure by using the SAT/ACT scores. 

Given the exploratory status of the 56 items (see Table 2.8 for example items), we examined 
the structure of the entire (synthetic) correlation matrix. From prior work on the structure of ability, 
we expected a hierarchical structure with several correlated primaries and a higher order g factor. 
This structure was confirmed by using the Very Simple Structure (Revelle & Rocklin, 1979) 
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Table 2.8 Example ability items with the number of items presented and an estimate of internal consistency (all items 
were multiple choice) 


Type 

N 

a 

Example item stem 

General knowledge 

1 

.46 

Tycho Brahe was a famous: 

Classification 

1 

.28 

Please mark the word that does not match the other words 

Pattern 

1 

.58 

In the following alphanumeric series, what letter comes next? 

Math reasoning 

1 

.63 

Adam and Melissa went fly-fishing and caught a total of 32 salmon 
Melissa caught three times as many salmon as Adam. 

How many salmon did Adam catch? 

Vocabulary 

6 

.35 

The opposite of an “ambiguous” situation is a/an (blank) situation 

Verbal analogy 

8 

.28 

CLOCK is to TIME as SCALE is to? 

Geometric analogy 

14 

.64 

See Fig. 2.1 


Table 2.9 The correlations of self-reported ability measures and unit weighted factor estimates. After the first kS.OOO 
participants, total SAT was replaced by the two subtests (SATV and SATQ), and thus there are no cross correlations 
between these measures, a reliabilities are reported on the diagonal for the factor estimates. Combined reflects unit 
weighted scores of the best 36 items, reasoning, spatial, and verbal are unit weighted scores of the sahent items on the 
corresponding oblique factors. The correlations between the combined score and the three factors are inflated due to 
item overlap. Correlations above the diagonal are corrected for attenuation. There is no correction for the reliabilities 
of education, age, or the SAT and ACT scores. Gender was coded 1 for M, 2 for F 



IVLT 

Edu 

Age 

SAT 

SATV 

SATQ 

ACT 

Reas 

Spat 

Verb 

Comb 

Gender 

1.00 

0.03 

0.01 

-0.08 

0.01 

-0.12 

-0.04 

-0.15 

-0.09 

0.16 

-0.09 

Education 

0.03 

1.00 

0.45 

0.02 

0.11 

0.09 

0.12 

0.35 

0.25 

0.03 

0.23 

Age 

0.01 

0.45 

1.00 

-0.04 

0.04 

0.01 

0.12 

0.24 

0.09 

0.03 

0.09 

SAT 

-0.08 

0.02 

-0.04 

1.00 

NA 

NA 

0.65 

0.48 

0.32 

0.13 

0.31 

SATV 

0.01 

0.11 

0.04 

NA 

1.00 

0.62 

0.54 

0.50 

0.25 

0.12 

0.25 

SATQ 

-0.12 

0.09 

0.01 

NA 

0.62 

1.00 

0.57 

0.50 

0.38 

0.10 

0.36 

ACT 

-0.04 

0.12 

0.12 

0.65 

0.54 

0.57 

1.00 

0.55 

0.35 

0.13 

0.34 

Reasoning 

-0.13 

0.30 

0.21 

0.41 

0.43 

0.43 

0.47 

0.73 

0.75 

0.43 

0.77 

Spatial 

-0.08 

0.22 

0.08 

0.28 

0.22 

0.33 

0.30 

0.56 

0.77 

0.38 

1.25 

Verbal 

0.10 

0.02 

0.02 

0.08 

0.08 

0.06 

0.08 

0.24 

0.22 

0.42 

0.92 

Combined 

-0.08 

0.27 

0.15 

0.37 

0.34 

0.40 

0.41 

0.84 

0.88 

0.45 

0.83 


criterion which showed a clear one factor solution for complexity one, and a three factor solution 
for complexity two. The cOj^ value when three lower order factors were extracted was .64. The alpha 
value for the entire 56 items was .88, and the was .89. See Revelle & Zinbarg (2009) for the 
meaning of these three and other estimates of reliability. The 14 salient items on the first factor were 
a mix of reasoning and vocabulary items, the 18 salients on the second factor were seven alphanu¬ 
meric series questions and 11 spatial analogies, and the six salients on the third factor were verbal 
logic items. The salient items on the g factor as extracted using either omega (38 items) or ICLUST 
(37 items) were chiefly a mix of items from the first two factors. 

Internal structure is not enough to show the utility of these factors. Unit weighted scales were 
used to predict real world criteria such as education and age as well as the self-reported test scores 
(Table 2.9). The independent contribution of the three oblique ability factors for predicting the 
criteria may be seen in a set of multiple regressions (Table 2.10). It is clear that for research pur¬ 
poses, we can use short online ability measures to predict level of education or standardized ability 
measures. It is interesting that the short reasoning factor (assessed with just 14 items and an a of 
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Table 2.10 Regressions predicting demographic and test performance from the SAPA ability 
factors. Values are standardized beta weights and multiple Rs 



Gender 

Education 

Age 

SAT 

SATV 

SATQ 

ACT 

Reasoning 

-0.15 

0.28 

0.25 

0.39 

0.47 

0.38 

0.45 

Spatial 

-0.03 

0.08 

-0.04 

0.08 

-0.04 

0.13 

0.07 

Verbal 

0.14 

-0.05 

-0.03 

-0.05 

-0.02 

-0.06 

-0.03 

Multiple R 

0.19 

0.32 

0.22 

0.43 

0.44 

0.46 

0.48 


Fig. 2.2 Using a Schmid 
Leiman transformation, the 
structure of the self 
reported ability tests and 
the web based assessment 
can be revealed with a com¬ 
mon general factor as well 
as two orthogonal residual 
factors 


Schmid Leiman solution of ability measures 



just .73), is such a good measure. The Spatial factor, although not the best predictor of any of the 
standardized tests, is clearly assessing a component of ability not tapped by the reasoning factor. 

The hierarchical structure of the self-reported ability measures and the online ability measures 
can be seen graphically by plotting the results of a Schmid-Leiman transformation (Schmid & Leiman, 
1957) to extract a general factor and two orthogonal residual factors (Fig. 2.2). The residual factors 
clearly represent method: one being self-reported ability, the second being our online (and shorter 
and thus less reliable) measures. 

The relationships can be examined in more detail in two ways: (1) what is the effect of the demo¬ 
graphic and ability measures on Big 5 scales (Table 2.13) and (2) what is the effect of the Big 5 on 
demographic and ability measures (Table 2.14). In two sets of hierarchical regressions, it is clear 
that the Big 5 scales are systematically related to Gender (Stability and Agreeableness), to age and 
education (Conscientiousness), and to intellectual ability, assessed either by our new procedures, or 
conventional standardized tests (Openness). But these relations go both ways, for gender, education, 
age, and the ability scales all could be meaningfully predicted by the Big 5 measures (Table 2.13). 
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Table 2.11 Hierarchical regression predicting SAPA ability factors from self-reported 
standardized tests. Values are standardized beta weights and multiple Rs 



Combined 

Reasoning 

Spatial 

Verbal 

Step 1 

SATV 

0.12 

0.27 

0.00 

0.07 

SATQ 

0.34 

0.27 

0.33 

0.02 

Multiple R 

0.42 

0.49 

0.33 

0.08 

Step 2 

SATV 

0.04 

0.18 

-0.06 

0.05 

SATQ 

0.24 

0.16 

0.25 

-0.01 

ACT 

0.25 

0.29 

0.20 

0.07 

Multiple R 

0.46 

0.54 

0.36 

0.10 


Table 2.12 Correlations between the Big 5 measures, demographics, and ability measures. 
Reliabilities for the Big 5 are shown in the appropriate diagonal 



Extra 

Stability 

Cons 

Agree 

Qpen 

Gender 

0.07 

-0.20 

0.13 

0.25 

-0.10 

Education 

0.00 

0.05 

0.18 

0.10 

0.16 

Age 

-0.01 

0.09 

0.20 

0.10 

0.13 

SAT 

-0.11 

0.02 

-0.08 

-0.14 

0.25 

SATV 

-0.07 

0.02 

-0.08 

-0.05 

0.33 

SATQ 

-0.05 

0.09 

-0.02 

-0.08 

0.23 

ACT 

-0.05 

0.04 

-0.01 

-0.06 

0.30 

Combined 

-0.08 

0.10 

0.00 

0.00 

0.28 

Reasoning 

-0.08 

0.09 

-0.02 

-0.03 

0.30 

Spatial 

-0.07 

0.09 

0.01 

0.00 

0.20 

Verbal 

0.04 

0.02 

0.08 

0.12 

0.10 

Extraversion 

0.93 

0.28 

0.14 

0.41 

0.30 

Stability 

0.28 

0.93 

0.17 

0.17 

0.17 

Conscientiousness 

0.14 

0.17 

0.92 

0.25 

0.13 

Agreeableness 

0.41 

0.17 

0.25 

0.90 

0.21 

Openness 

0.30 

0.17 

0.13 

0.21 

0.83 


Study 7: Integrating Cognitive and Non-cognitive Measures of Personality: The “Big 5” 

Meet IQ 

To examine the relationship of “non-cognitive” and “cognitive” aspects of personality we correlated 
the Big 5 composite scores with the ability scores discussed in Study 6. We did this for the >66,000 
participants with Big 5 and IQ items. Each of the Big 5 composite scales was estimated by using 20 
composite items. The IQ factors were estimated hy unit weighted composites of the salient items. 
In addition to these measures, we also examined the relationships with the demographic variables 
of age, gender, and education (Table 2.12). These zero order correlations show that gender (male = 1, 
female = 2) was positively correlated with Agreeableness (.25) and negatively correlated with 
Emotional Stability (-.20). Older and more educated participants were more Conscientious (.18 and 
.20) and more Open (.13 and .16). Of the Big 5, Openness was most related to the ability measures 
(.23<r<.33). The correlation of ability with openness partly reflects the emphasis on “intellect” in 
the choice of the openness scales from the IPIP (Tables 2.13 and 2.14). 
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Table 2.13 Hierarchical regressions predicting Big 5 measures from demographic and ahihty 
measures. Values are standardized beta weights and multiple Rs 



Extra 

Stability 

Cons 

Agree 

Open 

Step 1 

Gender 

0.07 

-0.20 

0.12 

0.25 

-0.11 

Education 

0.00 

0.02 

0.11 

0.05 

0.13 

Age 

-0.01 

0.08 

0.15 

0.07 

0.07 

Multiple R 

0.07 

0.22 

0.26 

0.27 

0.20 

Step 2 

Gender 

0.05 

-0.20 

0.10 

0.23 

-0.08 

Education 

0.03 

0.01 

0.13 

0.07 

0.06 

Age 

-0.01 

0.08 

0.16 

0.08 

0.05 

Reasoning 

-0.07 

0.01 

-0.11 

-0.07 

0.23 

Spatial 

-0.05 

0.06 

0.02 

0.01 

0.04 

Verbal 

0.06 

0.03 

0.08 

0.11 

0.04 

Multiple R 

0.12 

0.23 

0.28 

0.29 

0.32 

Step 3 

0.05 

-0.19 

0.11 

0.23 

-0.10 

Education 

0.03 

0.01 

0.13 

0.07 

0.07 

Age 

-0.01 

0.08 

0.16 

0.08 

0.05 

Reasoning 

-0.05 

0.00 

-0.09 

-0.03 

0.08 

Spatial 

-0.06 

0.05 

0.00 

0.01 

0.05 

Verbal 

0.06 

0.03 

0.09 

0.10 

0.05 

SATV 

-0.06 

-0.04 

-0.13 

-0.02 

0.24 

SATQ 

0.03 

0.08 

0.09 

-0.02 

-0.07 

ACT 

0.00 

-0.02 

0.02 

-0.05 

0.13 

Multiple R 

0.13 

0.24 

0.30 

0.30 

0.41 


Table 2.14 Hierarchical regression predicting demographic and ability measures from the 
Big 5 measures. Values i 1 d ^ed beta weights and multiple Rs. Note that the Giant 2 

of Extraversion and Emotional Stability have the weakest effects in predicting demographic 
or ability measures 


Gender 

Edu 

Age 

SAT 

SATV 

SATQ 

ACT 

Step 1 

Extraversion 

0.14 

-0.02 

-0.04 

-0.13 

-0.08 

-0.08 

-0.07 

Stability 

-0.24 

0.05 

0.10 

0.06 

0.04 

0.11 

0.06 

Multiple R 

0.24 

0.05 

0.10 

0.12 

0.08 

0.12 

0.08 

Step 2 

Extraversion 

0.12 

-0.03 

-0.06 

-0.12 

-0.07 

-0.08 

-0.07 

Stability 

-0.26 

0.03 

0.07 

0.07 

0.05 

0.12 

0.06 

Conscientiousness 

0.16 

0.18 

0.20 

-0.07 

-0.08 

-0.03 

-0.03 

Multiple R 

0.28 

0.18 

0.22 

0.14 

0.11 

0.12 

0.08 

Step 3 

Extraversion 

0.06 

-0.10 

-0.11 

-0.15 

-0.16 

-0.11 

-0.13 

Stability 

-0.26 

0.01 

0.06 

0.04 

0.02 

0.10 

0.03 

Conscientiousness 

0.12 

0.15 

0.17 

-0.07 

-0.10 

-0.03 

-0.02 

Agreeableness 

0.27 

0.07 

0.07 

-0.13 

-0.05 

-0.10 

-0.08 

Openness 

-0.15 

0.15 

0.12 

0.33 

0.40 

0.27 

0.35 

Multiple R 

0.39 

0.24 

0.25 

0.35 

0.39 

0.29 

0.34 
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Summary and Conclusions 

The proper study of personality integrates affect, behavior, cognition, and desires. But to study all 
of these aspects at the same time would seem to require large samples of participants taking many 
different instruments. We have an introduced an alternative procedure, Synthetic Aperture 
Personality Assessment, which allows us to combine data from many different individuals taking 
overlapping but non-identical surveys. These techniques have been used to explore structural ques¬ 
tions about cognitive and non-cognitive aspects of personality as well as to explore the link between 
these two aspects of an individual. Although some of the results are merely demonstrations of the 
technique, others provide greater insight into the structure of cognitive ability and non-cognitive 
sensitivities. The goal of the SAPA project is to allow others to take advantage of these open source 
procedures and to proceed to build a greater understanding of personality structure and processes. 
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